feat[bc|notask]: migrate AI SDK provider to v7 by danielAsaboro · Pull Request #3370 · tetherto/qvac

danielAsaboro · 2026-07-21T10:41:27Z

Summary

This migrates @qvac/ai-sdk-provider from AI SDK 6/provider v3 to AI SDK 7/provider v4.

It preserves uploaded MIME types in the CLI ephemeral-file store and exposes the same AI SDK 7 provider capabilities in external and managed modes.

API usage

The callable provider is shorthand for selecting a language model. The explicit language-model methods remain available, and other model types use their capability-specific methods:

import { qvac } from '@qvac/ai-sdk-provider'

const language = qvac('qwen3-600m')
const explicitLanguage = qvac.chatModel('qwen3-600m')

const embedding = qvac.textEmbeddingModel('embed-gemma')
const image = qvac.imageModel('flux-schnell')
const transcription = qvac.transcriptionModel('whisper')
const speech = qvac.speechModel('tts')
const files = qvac.files()

These methods select model aliases served by QVAC; they do not load every capability through the callable language-model shorthand.

Managed mode exposes the same surface after starting the configured local models:

import { createQvac } from '@qvac/ai-sdk-provider'

const qvac = await createQvac({
  mode: 'managed',
  models: ['QWEN3_600M_INST_Q4', 'EMBEDDINGGEMMA_300M_Q4_0']
})

const language = qvac('QWEN3_600M_INST_Q4')
const embedding = qvac.embeddingModel('EMBEDDINGGEMMA_300M_Q4_0')

Changes

Migrates @qvac/ai-sdk-provider to AI SDK 7, @ai-sdk/openai-compatible 3, provider v4 contracts, Node 22+, and ESM-only operation.
Keeps qvac(modelId) as the language-model shorthand while retaining languageModel() and chatModel().
Adds native files, transcription, and speech adapters.
Preserves the complete language, embedding, image, files, transcription, and speech surface in external and managed modes.
Resolves QVAC file provider references through the caller-managed local serve before language-model requests.
Uses provider-utils response handlers so audio, upload, and file-reference failures expose structured APICallError metadata.
Preserves multipart MIME types in the CLI ephemeral-file store and returns them from /v1/files/:id/content.
Normalizes request headers and consistently forwards caller authorization and per-call headers.
Runs the AI SDK v7 codemod as a migration audit, followed by manual review and formatting.

Behavior / compatibility

Breaking: @qvac/ai-sdk-provider now requires Node.js 22 or newer, AI SDK 7, and provider v4 packages.
Breaking: the package is ESM-only; CommonJS require() is not supported.
No AI SDK 6 compatibility layer or deprecated duplicate provider API was added.
qvac('alias'), qvac.languageModel('alias'), and qvac.chatModel('alias') select the same language-model capability.
Embedding and image selection remain explicit through embeddingModel() / textEmbeddingModel() and imageModel().
Transcription and speech use transcriptionModel() and speechModel().

Package requirement change:

BEFORE:

{
  "engines": { "node": ">=20.0.0" },
  "peerDependencies": {
    "ai": "^6.0",
    "@ai-sdk/openai-compatible": "^2.0"
  }
}

AFTER:

{
  "engines": { "node": ">=22.0.0" },
  "peerDependencies": {
    "ai": "^7.0",
    "@ai-sdk/openai-compatible": "^3.0"
  }
}

Testing

Provider: format; lint with zero warnings; typecheck; build; 81 unit tests passed and 2 managed integration tests skipped; package dry-run and ESM export smoke passed.
Provider behavior: language, embedding, and image methods; callable and explicit language-model selectors; file upload/reference round trip; structured error metadata; header propagation; speech and transcription request/response handling; managed-mode surface.
CLI: format; lint; typecheck; build; all 417 unit tests passed.
CLI benchmark harness: benchmark typecheck and all 85 harness tests passed.
CLI files HTTP suite: all 7 tests passed, covering exact bytes, MIME header, private cache control, metadata, listing, and missing files.

The request-scoped tools_compact / dynamic-tools changes were removed from this PR following maintainer feedback. This PR no longer changes the SDK or llm-llamacpp addon.

github-actions · 2026-07-21T10:41:38Z

Review Status

Current Status: ✅ APPROVED
Approvals so far: Team Lead: 1, Member: 1

simon-iribarren

This is a breaking public change, so the title should use [bc] rather than [api|bc]. Please also document the Node 22+, AI SDK 7/provider-v4, and ESM-only requirements in the README. Full CI is currently skipped because the PR lacks the verified label.

Copilot

Pull request overview

This PR upgrades @qvac/ai-sdk-provider to AI SDK 7 / provider v4, expands the provider surface to include native files + speech + transcription, and fixes a llama.cpp dynamic-tools regression by making tools_compact a per-request decision (explicitly false for toolless prompts) across single and batch completions. It also preserves uploaded MIME types in the CLI ephemeral-file store.

Changes:

Migrate the AI SDK provider package to AI SDK 7/provider v4 (Node 22+, ESM-only) and compose native QVAC files/transcription/speech capabilities over the OpenAI-compatible transport.
Make tools_compact request-scoped in the SDK and addon (single + continuous batch), including per-request enable/disable and safe handling for toolless prompts.
Preserve multipart MIME types through the CLI ephemeral file store and validate via HTTP e2e tests.

Reviewed changes

Copilot reviewed 36 out of 37 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/sdk/test/unit/tools-compact.test.ts	Adds unit coverage for request-scoped `tools_compact` computation and mixed-batch behavior.
packages/sdk/test/unit/llm-plugin-transform.test.ts	Asserts `tools_compact` is not set at model-load transform time.
packages/sdk/test/bare/tools-compact.test.ts	Bare-runtime regression test ensuring toolless dynamic requests succeed with `tools_compact: false`.
packages/sdk/server/bare/plugins/llamacpp-completion/transform.ts	Removes model-load-time mapping from tools mode to `tools_compact`.
packages/sdk/server/bare/plugins/llamacpp-completion/ops/tools-compact.ts	Introduces shared helpers for computing/merging request-scoped `tools_compact`.
packages/sdk/server/bare/plugins/llamacpp-completion/ops/completion-stream.ts	Applies request-scoped `tools_compact` in single completion requests and preserves other per-request params.
packages/sdk/server/bare/plugins/llamacpp-completion/ops/batch-completion-stream.ts	Applies request-scoped `tools_compact` per batch slot and always forwards generation params.
packages/sdk/schemas/tools.ts	Updates docs to reflect request-scoped `tools_compact` behavior for dynamic tools.
packages/llm-llamacpp/test/unit/test_tools_compact_controller.cpp	Extends controller unit tests for enable/disable and unsupported behavior.
packages/llm-llamacpp/test/integration/tools-compact.test.js	Adds integration coverage for per-request override and mixed-slot continuous batch isolation.
packages/llm-llamacpp/test/integration/generation-params.test.js	Adds validation that `tools_compact` overrides must be boolean at the JS boundary.
packages/llm-llamacpp/index.js	Enforces runtime type checking for `generationParams.tools_compact`.
packages/llm-llamacpp/index.d.ts	Exposes `tools_compact?: boolean` as a per-request generation param override.
packages/llm-llamacpp/addon/src/model-interface/ToolsCompactController.hpp	Enables controller to start disabled and supports per-request toggling.
packages/llm-llamacpp/addon/src/model-interface/ToolsCompactController.cpp	Implements `setEnabled()` and ensures disabling clears boundaries and validation is gated by enabled state.
packages/llm-llamacpp/addon/src/model-interface/TextLlmContext.hpp	Adds virtual hook to toggle tools compaction per request.
packages/llm-llamacpp/addon/src/model-interface/TextLlmContext.cpp	Rebuilds chat templates when the effective tools-compaction state changes.
packages/llm-llamacpp/addon/src/model-interface/MtmdLlmContext.hpp	Adds per-request tools-compaction toggle for multimodal contexts.
packages/llm-llamacpp/addon/src/model-interface/MtmdLlmContext.cpp	Mirrors template rebuild logic for multimodal contexts.
packages/llm-llamacpp/addon/src/model-interface/LlmContext.hpp	Adds `tools_compact` override plumbing + a dedicated setter separate from sampler overrides.
packages/llm-llamacpp/addon/src/model-interface/LlamaModel.hpp	Tracks a load-time default and supports request overrides.
packages/llm-llamacpp/addon/src/model-interface/LlamaModel.cpp	Applies per-request `tools_compact`, logs unsupported overrides, and wires defaults into schedulers.
packages/llm-llamacpp/addon/src/model-interface/ContinuousBatchScheduler.hpp	Carries a tools-compaction default through the scheduler.
packages/llm-llamacpp/addon/src/model-interface/ContinuousBatchScheduler.cpp	Instantiates per-seq tools controllers with request-specific enabled state and warnings on unsupported models.
packages/llm-llamacpp/addon/src/addon/AddonJs.hpp	Parses `tools_compact` as an optional boolean generation override from JS.
packages/cli/test/e2e/http/files-validation.test.ts	Verifies `/v1/files/:id/content` returns the original uploaded `Content-Type`.
packages/cli/src/serve/routes/files.ts	Stores upload MIME type in the ephemeral file record.
packages/ai-sdk-provider/test/provider.test.ts	Expands provider tests for v4 surface, local audio endpoints, files upload/reference resolution, and APICallError shape.
packages/ai-sdk-provider/test/managed-provider.test.ts	Verifies managed-mode provider exposes the same expanded surface.
packages/ai-sdk-provider/src/types.ts	Updates provider types to include files/transcription/speech v4 contracts.
packages/ai-sdk-provider/src/provider.ts	Composes custom provider capabilities and wraps language models for local file reference resolution.
packages/ai-sdk-provider/src/managed/index.ts	Reuses `createExternalQvac` for managed-mode provider construction.
packages/ai-sdk-provider/src/files.ts	Implements AI SDK v4 Files adapter backed by QVAC serve’s ephemeral store.
packages/ai-sdk-provider/src/file-reference-model.ts	Resolves `{ qvac: <id> }` file references via local `/files/:id/content` before model calls.
packages/ai-sdk-provider/src/audio-models.ts	Adds native transcription + speech model adapters targeting QVAC local audio endpoints.
packages/ai-sdk-provider/README.md	Updates docs for Node 22+, AI SDK 7/provider v4, and new files/audio capabilities.
packages/ai-sdk-provider/package.json	Bumps engine/peer deps and adds provider/provider-utils runtime deps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

danielAsaboro · 2026-07-22T16:26:12Z

Addressed all review feedback in d58e8851 and 3a5fe82b.

Added native per-request tools_compact support across the JS boundary, serialized requests, and isolated continuous-batch slots. Load-time configuration remains the default for direct addon consumers.
Added validation and real-addon coverage for request overrides, explicit disable/reset, and mixed enabled/disabled batches.
Replaced plain HTTP errors in audio, uploads, and file-reference handling with structured APICallError responses via provider-utils.
Documented Node 22+, AI SDK 7, provider-v4, and ESM-only requirements.
Updated the external-provider comment to describe its native files/audio capabilities and file-reference resolution accurately.
Updated the PR title to use [bc].

Verification passed locally:

Provider format, lint, typecheck, build, unit tests, and packed ESM smoke
SDK format, lint, typecheck, build, unit, and Bare tests
Native addon build and type checks
C++ tools-compaction tests: 41/41
Generation-parameter integration: 3/3
Real-addon tools-compaction integration: 21/21

Could a maintainer please apply verified, run-cpp-addon-tests, and run-desktop-addon-tests so the full native CI suite runs?

Copilot

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

packages/ai-sdk-provider/src/provider.ts:20

The comment says external mode is kept “byte-for-byte identical” to the v1 behavior, but this function now composes additional native capabilities (files/transcription/speech) and wraps language models to resolve local file references. This documentation is now misleading for readers trying to understand what differs from the older wrapper.

// External mode synchronously composes the OpenAI-compatible language-model
// surface with QVAC's native files, transcription, and speech capabilities.
// Language models also resolve QVAC file references through the caller-managed
// `qvac serve openai` endpoint.

Copilot

Pull request overview

Copilot reviewed 36 out of 37 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

packages/ai-sdk-provider/src/provider.ts:25

createExternalQvac can end up sending two Authorization headers if the caller passes headers: { authorization: ... } (lowercase) and this code also adds Authorization (capitalized). Because header names are case-insensitive, this can lead to an ambiguous/combined Authorization value and break authentication. Normalize by deleting any pre-existing authorization header (any casing) and then setting exactly one authorization header based on the resolved apiKey.

  const headers: Record<string, string> = { ...DEFAULT_HEADERS, ...options.headers }
  if (options.apiKey ?? DEFAULT_API_KEY) {
    headers['Authorization'] = `Bearer ${options.apiKey ?? DEFAULT_API_KEY}`
  }

Copilot

Pull request overview

Copilot reviewed 37 out of 38 changed files in this pull request and generated no new comments.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

opaninakuffo · 2026-07-23T11:58:34Z

hey @danielAsaboro, thanks for the AI SDK v7 work!

one thing, the request-scoped tools_compact / dynamic-tools fix overlaps with work we're landing to remove SDK dynamic tools mode and the addon tools_compact path (#3380, #3373). that surface is being deprecated, so we'd rather not deepen it here.

could you kindly carve those bits out and keep this PR focused on the AI SDK 7 migration?

also curious, were you relying on dynamic tools or did you just hit the bug along the way?

happy to re-review once it's split

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

danielAsaboro · 2026-07-23T14:28:10Z

Hi, @opaninakuffo

Thanks for the context ...that makes sense.

I was relying on dynamic tools in Leash rather than just encountering the code incidentally.

We serve Qwen3 with tools: true / toolsMode:"dynamic" and expose a focused, per-turn subset of a larger tool registry.

The regression surfaced on real tool-less paths - including raw completions, mesh-forward/mobile borrowing, and reasoning-only subagents. This is often where a model loaded in dynamic mode received no tools and failed
with tools_compact requires non-empty tools.

I implemented the request-scoped fix before I was aware that #3373 and #3380 were removing the SDK and addon surfaces entirely. Given that direction, I agree it doesn’t make sense to deepen the feature here.

I’ll carve all tools_compact / dynamic-tools changes and their tests out of #3370, update the title and description, and keep the PR focused on the AI SDK 7/provider-v4 migration. I’ll rerun the relevant provider and CLI checks and request another review afterward.

On the Leash side, I’ll retain the existing 0.13.x workaround while needed and migrate to static tool placement when the removal ships. Thank you

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

victorchimakanu · 2026-07-24T14:22:24Z

Two items from Simon's review still outstanding before CI can run fully:

Title — change [api|bc] to [bc] (the api tag is not needed here since this is a breaking change to an existing API, not a new surface).
README — add a note on the Node 22+, AI SDK 7 / provider-v4, and ESM-only requirements.

The verified label also needs to be applied by a team member to unlock full CI — that will follow once the above are addressed.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

opaninakuffo

thanks for the context and for carving it out quickly. Sounds good on Leash and the pr lgtm!

Copilot

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

danielAsaboro · 2026-07-24T18:39:35Z

hi @opaninakuffo, no new changes was made other than pulling from the upstream main
(saw that the verified label was removed)

github-actions Bot added the community-contribution PR or issue from an external contributor label Jul 21, 2026

danielAsaboro marked this pull request as ready for review July 21, 2026 11:03

danielAsaboro requested review from a team as code owners July 21, 2026 11:03

simon-iribarren reviewed Jul 22, 2026

View reviewed changes

Comment thread packages/sdk/server/bare/plugins/llamacpp-completion/ops/tools-compact.ts Outdated

Comment thread packages/ai-sdk-provider/src/audio-models.ts Outdated

danielAsaboro changed the title ~~feat[api|bc|notask]: migrate AI SDK provider to v7 and fix dynamic tools~~ feat[bc|notask]: migrate AI SDK provider to v7 and fix dynamic tools Jul 22, 2026

Copilot AI review requested due to automatic review settings July 22, 2026 16:16

Copilot started reviewing on behalf of danielAsaboro July 22, 2026 16:16 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread packages/ai-sdk-provider/src/provider.ts Outdated

Copilot AI review requested due to automatic review settings July 22, 2026 16:23

Copilot started reviewing on behalf of danielAsaboro July 22, 2026 16:23 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread packages/ai-sdk-provider/src/provider.ts Outdated

Comment thread packages/ai-sdk-provider/src/file-reference-model.ts

Copilot AI review requested due to automatic review settings July 22, 2026 16:30

Copilot started reviewing on behalf of danielAsaboro July 22, 2026 16:30 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 22, 2026 16:53

Copilot started reviewing on behalf of danielAsaboro July 22, 2026 16:53 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 22, 2026 17:04

Copilot AI reviewed Jul 22, 2026

Copilot AI review requested due to automatic review settings July 22, 2026 18:51

Copilot AI reviewed Jul 22, 2026

Copilot AI review requested due to automatic review settings July 23, 2026 14:23

Copilot AI reviewed Jul 23, 2026

Copilot AI review requested due to automatic review settings July 24, 2026 13:33

Copilot AI reviewed Jul 24, 2026

danielAsaboro changed the title ~~feat[bc|notask]: migrate AI SDK provider to v7 and fix dynamic tools~~ feat[bc|notask]: migrate AI SDK provider to v7 Jul 24, 2026

danielAsaboro added 2 commits July 24, 2026 15:27

feat[bc]: migrate AI SDK provider to v7

a786e80

fix: preserve uploaded file content type

eda0f9f

Copilot AI review requested due to automatic review settings July 24, 2026 14:29

danielAsaboro force-pushed the feature-ai-sdk-v7 branch from 8f64f39 to eda0f9f Compare July 24, 2026 14:29

Copilot AI reviewed Jul 24, 2026

opaninakuffo added the verified Authorize secrets / label-gate in PR workflows label Jul 24, 2026

opaninakuffo approved these changes Jul 24, 2026

View reviewed changes

simon-iribarren approved these changes Jul 24, 2026

View reviewed changes

Merge branch 'main' into feature-ai-sdk-v7

3ae6552

Copilot AI review requested due to automatic review settings July 24, 2026 16:04

Copilot AI reviewed Jul 24, 2026

github-actions Bot removed the verified Authorize secrets / label-gate in PR workflows label Jul 24, 2026

opaninakuffo added the verified Authorize secrets / label-gate in PR workflows label Jul 24, 2026

Merge branch 'main' into feature-ai-sdk-v7

3ac1766

Copilot AI review requested due to automatic review settings July 24, 2026 18:22

Copilot AI reviewed Jul 24, 2026

github-actions Bot removed the verified Authorize secrets / label-gate in PR workflows label Jul 24, 2026

Uh oh!

Conversation

danielAsaboro commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

API usage

Changes

Behavior / compatibility

Testing

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review Status

Uh oh!

simon-iribarren left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

danielAsaboro commented Jul 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

opaninakuffo commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

danielAsaboro commented Jul 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

victorchimakanu commented Jul 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

opaninakuffo left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

danielAsaboro commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

danielAsaboro commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

opaninakuffo commented Jul 23, 2026 •

edited

Loading